Introduction to Differential Equations

Since Newton, mankind has come to realize that the laws of physics are always expressed in the language of differential equations.

Steven Strogatz

Differential equations are equations that involve an unknown function and its derivatives. They are used to model a wide variety of phenomena in the natural and social sciences. We will introduce the basic concepts of differential equations and explore some simple examples.

Note that this is just an introduction to differential equations. The topic is vast and deep, and there are many advanced topics that we will not cover here.

Some advanced topics not covered here include:

• Partial differential equations
• The Laplace transform
• The Fourier transform
• DEs involving complex numbers

Table of Contents

• Table of Contents
• What is a Differential Equation?
  • Example Problem: Formulating The Differential Equation for Radioactive Decay
• Verifying Solutions to Differential Equations
  • Example Problem: Verifying a Solution
• Solving Differential Equations
• Slope Fields
  • Example Problem: Equation from Slope Field
  • Example Problem: Slope Field from Equation
• Qualitatively Analyzing Slope Fields to Approximate Solution Curves
  • Example Problem: Range of Solution Curve
• Approximation Method: Euler's Method
  • Example Problem: Working with Euler's Method
  • Computing Approximations with Technology
• Separation of variables
  • Separable Differential Equations
  • Note: Algebraic Manipulation of Differentials
  • Identifying Separable Differential Equations
• Exponential Models
  • Example Problem: Radioactive Decay
  • Example Problem: Newton's Law of Cooling
• Logistic Growth Models
  • Properties of the Logistic Growth Model
  • Solving the Logistic Growth Model
  • Example Problem: Designing a RPG Defense System
• Homogeneous Differential Equations
  • Example Problem: Homogeneous Differential Equation
  • Example Problem: A More Complex Homogeneous Differential Equation
• Predator-Prey Models: Systems of Differential Equations
  • Interpreting the Predator-Prey Model
• The Simple Pendulum
• The Pitfall of Numerical Solutions
• Summary

What is a Differential Equation?

Firstly, let's consider a normal algebraic equation:

Here, you want to solve for the value of that satisfies the equation.

This is a differential equation:

Instead of solving for some variable, you solve for the function that satisfies the equation.

Example Problem: Formulating The Differential Equation for Radioactive Decay

Radioactive Decay is a process in which unstable atomic nuclei lose energy by emitting radiation.

Let be the number of radioactive atoms at time . The rate at which the number of atoms decays is proportional to the number of atoms present.

Write a differential equation that models the decay of radioactive atoms.

It's beneficial to first write a word equation that describes the problem:

We set as the proportionality constant. Now let's consider what each term should be:

• The rate of change of atoms is the derivative of the number of atoms with respect to time: .
• The number of atoms is .

Therefore, the differential equation is:

A visualization of the decay of radioactive atoms is shown below:

The arrow represents the rate of change of the number of radioactive atoms at time .

As time passes, the number of radioactive atoms decrease, and the rate at which they decay decreases as well.

Verifying Solutions to Differential Equations

To verify that a function satisfies a differential equation, just like verifying a solution to an algebraic equation, you substitute the function into the equation and check if it holds.

For example, consider the same differential equation from the previous example:

Let's verify that the function satisfies the equation:

The function satisfies the differential equation.

Example Problem: Verifying a Solution

Verify that the function satisfies the following differential equation:

Substitute into the equation:

The function does not satisfy the differential equation.

Solving Differential Equations

It's important to note this absolute fact:

Differential equations are very, very, very hard to solve.

In school, most courses focus on developing analytical solutions to DEs. This is, presumably, because by developing a solution, then you can get an intuition for the problem, as well as perform computations.

However, in the real world, most DEs either do not have an analytical solution or the solution is too complex to be useful

If you do not have a solution, this does not mean that you can't get an intuition or perform computations.

One way to perform computations is to numerically approximate the DE. Essentially, you get the initial state of the system, then use the DE to predict the change in the system over a small time interval. This contrasts the infinitesimal approach of calculus, where you find the exact change in the system at a point, which is why this is an approximation.

There are many ways to develop intuitions for DEs without solving them, which will be discussed later.

Slope Fields

One way to develop an intuition for DEs is to use slope fields. They are a visual representation of a DE that shows the slope of the solution at any point. Consider this DE:

Now we construct a graph. For each point , find the slope using the DE, then draw a small line segment with that slope at that point.

We can first construct a table like this:

For , the slope is , which is undefined. This likely means that it's a vertical line. We'll ignore this point for now. Doing this for all points from to gives us the following:

Using slope fields, we can get a general idea of what the solution to a DE looks like without actually solving it.

For instance, if we start at , we can see that the solution will move as follows:

The point you start at is called the initial condition. It is the starting point for the solution to the DE.

For example, in the radioactive decay problem, the initial condition could be the number of atoms at time , , or .

Example Problem: Equation from Slope Field

A differential equation has the following slope field:

Choose the correct differential equation from the following:

We can go through each option and see which one matches the slope field. To do so, we can pick a point that is easy to calculate the slope for, such as . At this point, looking visually, the slope is positive and more than .

1. :
   • For , the slope is , which is not correct.
2. :
   • For , the slope is , which is not correct.
3. :
   • For , the slope is , which is not correct.
4. :
   • For , the slope is , which could be correct.
5. :
   • For , the slope is , which is not correct.

Since is the only one that could be correct, it is, by elimination, the correct answer.

Therefore, the correct answer is .

Example Problem: Slope Field from Equation

A differential equation is given by:

Which of the following slope fields matches the differential equation?

Slope Field 1

Slope Field 2

Slope Field 3

Slope Field 4

Slope Field 5

Once again, we can pick a point, such as , and match the DE to the slope field.

At , the slope is . Therefore, the line should be flat at this point in the slope field.

Looking at the slope fields, we can see that the only one that has a flat line at is Slope Field 4.

Therefore, the correct answer is Slope Field 4.

Qualitatively Analyzing Slope Fields to Approximate Solution Curves

Slope fields can be used to approximate solution curves to DEs. Consider the DE:

The slope field looks like this:

We can trace a solution curve by starting at a point and following the slope field.

For example, let's start at . The slope at that point is .

It will roughly evolve like this:

The animation shows the solution curve evolving over time.

This, once again, shows that you can get an intuition for the solution to a DE without solving it.

Example Problem: Range of Solution Curve

Consider the DE from above.

Using the slope field, given the initial condition , what is the range of the solution curve?

Place the point on the slope field and follow the slope field to get an idea of the range of the solution curve.

Notice that the value slowly approaches , but never quite reaches it. Therefore, .

It never increases from the initial value of , so the range is below .

Thus, we can reasonably say that the range of the solution curve is .

Approximation Method: Euler's Method

Euler's Method is a numerical method to approximate the solution to a DE.

Consider the DE:

The exact solution to this DE is , but let's pretend we don't know that. Instead, what we do is start at the initial condition, , and follow the slope field, but this time, quantitatively instead of sketching it.

First, we need to choose a step size, . This is how much we move along the -axis each time. For this example, let's choose .

So we start at , where the slope is . This means that for every infinitesimal change, the slope is . We can use this to find the next point. Instead of an infinitesimal change, we make a finite change of .

This means that the next point is . Then we repeat the process.

It should look like this:

Options

Euler's method is one of the most simple and common methods to numerically approximate the solution to a DE. It's heavily used in computer simulations and is the basis for many other numerical methods.

For instance, consider a situation in which you are trying to simulate the motion of a particle in a fluid. This could be used for a game, or some type of scientific simulation. Since fluid dynamics is a very complex field, it's hard to get an analytical solution to the DEs that govern the motion of the particle. Instead, you can use Euler's method to approximate the motion of the particle.

As gets smaller, the approximation gets closer to the actual solution. This is similar to how the derivative is the limit of the slope as approaches .

Example Problem: Working with Euler's Method

Consider the DE:

Let , and let the increment be . After 3 steps, Euler's method yields the approximation . Express in terms of and .

Once again let's assume we don't know the solution to the DE.

We start at and follow the slope for . We can construct the following table:

Let's walk through the table.

• First, we set the initial condition on the first two columns.
• Then, using the DE in , we find the slope at to be .
• Finally, we find the change in by multiplying with , so it's .

For simplicity, let's denote . To understand this better, here's a visualization of it:

We can continue the table:

Let's break it down again:

• In the next increment, increases by , and increases by . Therefore, the new point is .
• Next, using those points, we find the slope at that point by plugging it into the DE in , yielding .
• To get the next , we multiply the slope by , giving . Denote this as .

Once again, let's visualize this:

Finally, we can continue with the third increment:

And visualize it:

So, we started at and after 3 steps, we reached .

We need to expand and :

So the value after 3 steps is:

Recall that the question stated that the approximation after 3 steps is . Setting , and then solving for gives:

Therefore, the answer is .

Computing Approximations with Technology

Euler's method is a very simple method to approximate the solution to a DE. However, it can be very tedious to do by hand, especially for more complex DEs or more steps. Thus, it's often done with technology. We'll use Python to demonstrate this, but this can be done with anything.

Consider the DE:

Let's approximate the solution after 300 steps with .

First, lay out the constants:

dx = 0.01
n = 300
x0 = 0
y0 = 1

Next, define the function for the slope:

def slope(x, y):
    return y ** 2 + x ** 2

Then, iterate through the steps:

x = x0
y = y0

for i in range(n):
    y += dx * slope(x, y)
    x += dx

Finally, print the result:

print(y)

This is the bare minimum to get the approximation. You can expand on this by:

• Storing the and values in a list.
• Using some plotting library to visualize the approximation.
• Using a loop to iterate through different step sizes or initial conditions to see how they affect the approximation.

Separation of variables

This is arguably one of the easiest and most common methods to solve a DE.

Separable Differential Equations

Consider the DE:

With a condition that the solution must pass through .

It might seem daunting at first, like as if you don't even know where to start. One approach you could try is to separate and to different sides.

Since we are able to separate and to different sides, this DE is known as a Separable Differential Equation.

Now, we can integrate both sides:

Remind me of the integration steps!

Recall that the derivative of is . So if we add a to the integrand, then we can perform a reverse chain rule:

Recall from the chain rule that the integral of is . Therefore:

We can combine the constants of integration into one constant, :

Next, we can use the initial condition to solve for the constant . Plugging and into the equation in :

Finally, we can substitute back into the equation in and solve for :

Since at , is positive, we can ignore the sign:

Therefore, the solution to the DE is .

To generalize this process, we can posit that a DE is separable if it can be written in the form:

Where is a function of and is a function of .

Note: Algebraic Manipulation of Differentials

When we separate variables, we treat differentials as if they were algebraic quantities.

Recall the formal definition of a derivative:

When we separate variables, we treat and as if they were algebraic quantities, and we can manipulate them as such. For instance:

While this seems reasonable, it's important to remember that differentials are not numbers, but rather infinitesimal quantities. Much real analysis is required to make this rigorous, but for the purposes of solving DEs, the notion of treating differentials as algebraic quantities is nonetheless a useful heuristic.

A more thorough treatment of differentials is beyond this scope, and is instead usually covered in a course on real analysis.

Identifying Separable Differential Equations

As mentioned, to identify a separable DE, look for a DE that can be written in the form:

One common type of DE that can be solved using separation of variables is one where is a product or quotient of functions of and :

Exponential Models

One extremely common DE describes exponential growth and decay. This was already mentioned above, but it's worth going into more detail.

Essentially, think of a quantity that, over time, either grows or shrinks at a rate proportional to its current size. For instance, interest in a bank account, the number of radioactive atoms in a sample, or the population of a species.

Let be the quantity at time . The general DE for exponential growth and decay is:

Where is a constant that depends on the specific situation. Usually it represents a rate of growth or decay.

In many cases, you want to start with a certain amount of the quantity at time , usually denoted as .

To solve this DE, we can make use of separation of variables:

We can integrate both sides spanning the interval;

• From to on the right side.
• When , and when , on the left side. So the bounds are and .

It's worth noting that appears in the solution. This is a result of a fundamental property of , which is that the derivative of is . This property is analogous to our case where the rate of change (derivative) of the quantity is proportional to the quantity itself. Always understand the meaning of each term in any equation outside of the algebra.

Example Problem: Radioactive Decay

Consider the radioactive decay of some substance. The rate of decay is proportional to the amount of the substance, where the constant of proportionality is , with SI units of . Let the amount of the substance at time be .

1. Write the DE that describes the decay.
2. Solve the DE with the initial condition .
3. The half-life, is defined as the time it takes for half of the substance to decay. Write an expression for the half-life in terms of , , and , as needed.
4. The half-life of is about 5730 years. Identify for .

1. We've already done the first part here.

2. Let's solve the DE:

3. The half-life is the time it takes for half of the substance to decay. We can formulate an equation to describe this:

   Substituting :

   This reveals a fundamental property of exponential decay: the half-life is independent of the initial amount of the substance. So whether you start with or of a substance, the time it takes for half of it to decay is the same.

4. The half-life of is about 5730 years. We can identify :

   Therefore, the constant of proportionality for is .

Example Problem: Newton's Law of Cooling

Another scenario where exponential models are used is in Newton's Law of Cooling.

Newton observed that the rate of cooling of an object is proportional to the difference in temperature between the object and its surroundings;

Where is the temperature of the object at time , is the temperature of the surroundings, and is a constant of proportionality.

First, we assume that is constant. This is a reasonable assumption for many scenarios, such as a cup of coffee cooling in a room; the room is so big that the cup won't warm the room up.

We can solve this DE using separation of variables:

Logistic Growth Models

Things can grow, but they can't grow infinite; there's always a limit to growth.

Thomas Malthus, an English cleric and scholar, observed that populations tend to grow exponentially, but they can't grow forever. He suggested that populations couldn't grow indefinitely due to limited resources, so we won't be able to produce infinite food to feed an infinite number of people, and the environment would impose a limit on growth.

Pierre François Verhulst, a Belgian mathematician, proposed a model that describes this type of growth. He suggested that the rate of growth of a population is proportional to the population itself, but it'll also have an asymptotic limit. This is contrary to Malthus' view that populations can go above the limit, but they'll crash and sort of oscillate around the limit.

Let be some quantity at time that grows according to the logistic model. To show how the logistic model works, we can start with the exponential growth model:

The graph of this function is an exponential curve that grows without bound. Let the limit of growth be , also known as the carrying capacity. So one way to model this is to dampen the growth rate as the population approaches the limit.

As , the growth rate should approach . We can introduce a term to dampen the growth rate:

The graph of this function is an S-shaped curve, known as the logistic curve. It looks like this (you can move around the point):

Note that this DE makes sense only under certain conditions:

• for all .
• .

Outside of these boundaries, the model might not make sense.

Properties of the Logistic Growth Model

It's really good to be able to understand certain properties of differential equations. They provide a very intuitive understanding of the system you're modeling, so it's a nice skill to be able to understand these properties without having to solve the DE. Furthermore, understanding these properties can help validate the DE to the real-world system you're modeling.

1. If , then for all . This makes sense because if you start with no population, then there's nothing to grow, so the population will remain at .
2. If , then for all . This makes sense because if you start with the carrying capacity, then the population is already at the limit, so it won't grow.

Let's also identify when the population is growing at its fastest rate. We can plot as a function of :

We treat as a function of and plot it. We can express as a parabola:

The maximum point of this parabola is the vertex, which is at . In addition, the rate is symmetric about the vertex, so the rate of growth is the same when the population is at as when it's at .

Solving the Logistic Growth Model

Recall that a DE is separable if it can be written in the form:

Let's apply this to the logistic growth model. Divide both sides by :

The left side is a bit tricky to integrate, but we can use partial fractions to simplify it:

Equating the numerators:

Substituting back into :

Substitute this back into :

We can drop the absolute value since the quantity inside is always positive:

Next, we can use the initial condition to solve for :

Substitute back into :

Example Problem: Designing a RPG Defense System

You are designing a defense system for a role-playing game. Players have a defense rating, and the higher the rating, the higher percentage of damage they can block.

Let be the percentage of damage blocked for a defense rating .

The conditions for the defense system are:

• The percentage should first increase rapidly, then slow down as the rating approaches .

• The percentage should never reach ; it should be an asymptote.

• Moreover, the defense system should also function for , which should increase the damage taken. The effects for negative ratings should be symmetric to the effects for positive ratings.

1. Construct a DE that describes the defense system.
2. Solve the DE with the initial condition .

I chose this problem:

1. To show that DEs can be used in many different contexts, not just in physics or biology.
2. Because I came across this exact problem myself when designing a game a few years ago.
3. To combine multiple concepts into one problem: the logistic growth model, initial conditions, and function transformations.

We can apply the logistic model to this problem, however, we need to make some modifications.

Let the carrying capacity, , be .

Recall that the growth rate is the fastest when , and the growth rate is symmetric about this point.

Given that we already solved the logistic model, we can use that as a base to solve this problem.

We can set to ensure that the growth rate is the fastest and is symmetric at . Furthermore, we can shift the curve down by to satisfy the second condition:

However, by shifting the curve down, we set the asymptotes to and . To ensure that the percentage approaches and instead, we can multiply the entire function by :

And we're done. We've successfully modeled a defense system for a role-playing game using a logistic growth model.

The graph of this function is an S-shaped curve that satisfies the conditions of the defense system:

Options

Homogeneous Differential Equations

(Not to be confused with homogeneous Linear DEs.)

Consider a DE of the form:

Let's assume we can't separate variables, and we can't solve it using the methods we've learned so far.

One clever trick exists if (and only if) the function can be written in a certain form:

This is known as a homogeneous differential equation. The reason it's useful is that we can make a substitution, namely , where is a new variable.

This substitution allows us to rewrite the DE in terms of and :

Remember that is dependent on , so to differentiate , we use the product rule:

So essentially, we have a DE in terms of and instead of and .

Next, we can use separation of variables to solve this DE:

From here, it depends on the specific form of .

Example Problem: Homogeneous Differential Equation

Solve the DE .

This DE is not separable, but it can be written in the form :

Next, we can make the substitution :

Example Problem: A More Complex Homogeneous Differential Equation

Solve the following DE:

Once again,

If , then , so we can drop the absolute value. We can write to simplify the expression:

Predator-Prey Models: Systems of Differential Equations

Predator-prey models are a type of DE that describe the interactions between two species: a predator and a prey.

The most famous predator-prey model is the Lotka-Volterra model. Let be the population of foxes (predators) and be the population of rabbits (prey).

We can think of the interactions between the two species as follows:

• Prey Abundance → Predator Growth - The more rabbits there are, the more food there is for the foxes, so the fox population increases.
• Predator Growth → Prey Decline - The more foxes there are, the more rabbits they eat, so the rabbit population decreases.
• Prey Decline → Predator Decline - The less rabbits there are, the less food there is for the foxes, so the fox population decreases.
• Predator Decline → Prey Abundance - The less foxes there are, the less rabbits are eaten, so the rabbit population increases.

So it sort of follows a cyclic pattern. We can create a system of two DEs to describe this. Let's consider the DE for the fox population first.

The rate of change of the fox population is proportional to the rabbit population (Prey Abundance → Predator Growth), as well as the fox population itself (Offspring).

However, foxes can also die due to natural causes, competition, or other predators, so we can introduce a term that dampens the growth rate.

As such, the DE for the fox population is:

Where represents growth from offspring and predation and represents death from external causes.

Next, let's consider the DE for the rabbit population.

It undergoes natural exponential growth proportional to its current population.

Just like foxes, the growth rate gets dampened by its own population. However, since rabbits are prey, the growth rate is also dampened by the fox population.

The DE for the rabbit population is:

Where represents growth from offspring and represents death from predation.

When there's a system of DEs, sometimes it's called a coupled system.

Interpreting the Predator-Prey Model

We can make use of various tools to interpret the predator-prey model without having to solve the DEs.

Recall that slope fields are used to visualize the behavior of a DE. In a system of DEs, we can plot the slope field by setting each axis to the population of one species. This is known as a phase space or phase portrait.

For the predator-prey model, we can plot the rabbit population on the -axis and the fox population on the -axis.

Options

From this visualization, we can see that the system has a stable equilibrium at two places:

• At the origin, where both species are extinct.
• At a point in the middle, where both species coexist and sort of oscillate around each other.

The Simple Pendulum

The simple pendulum is a mass (bob) attached to a string of length that swings back and forth. The angle that the string makes with the vertical is a function of time.

Since we are dealing with the pure mathematical model of the simple pendulum, we can assume the mass of the bob is and the string is massless.

We shall not consider the forces acting on the bob. For a more physical approach to this problem, see the Classical Mechanics section.

All we need to know is that the bob is subject to gravity, which acts downwards. The force of gravity can be broken down into two components: one parallel to the direction of motion and one perpendicular to it.

The component perpendicular to the direction of motion does not affect the motion of the bob. The component parallel to the direction of motion, which equals is the one that causes the bob to accelerate.

Dot Products

You can use something called the dot product here.

The dot product sort of "projects" one vector onto another. It's a way to find the component of one vector in the direction of another.

For two vectors and , the dot product is:

Therefore, we can write the component of in the direction of the bob as , where is a unit vector in the direction of the bob.

The acceleration can be written as:

Since , we can write the equation in terms of :

A more realistic model

Air drag is a force that opposes the motion of the bob. It is proportional to the velocity of the bob and acts in the opposite direction.

We can set the air drag force as , where is the proportionality constant.

The equation of motion becomes:

This DE, while more realistic, does not have an analytical solution.

Notice how this DE has a second derivative. Just like how the "order" of an algebraic equation is the highest power of the variable, the order of a differential equation is the highest derivative present.

This DE is a second-order DE because it has a second derivative.

This DE will be covered in more detail in the Classical Mechanics section.

The Pitfall of Numerical Solutions

As we know, many DEs cannot be solved analytically, so numerical methods are used to approximate the solution. There is a pitfall to numerical solutions that we need to be aware of:

Chaos.

Chaos is a phenomenon where a system is highly sensitive to initial conditions. This means that even a small change in the initial conditions can lead to a drastically different outcome. This is extremely dangerous in the context of numerical solutions because numerical methods are approximations.

One common example of chaos is the double pendulum, where two pendulums are attached to each other.

Summary

Differential equations are extremely powerful tools that can be used to model a wide variety of systems. The main steps within the study of DEs are:

• Formulating DEs - Understand the system you're modeling and formulate the DE accordingly.
• Solving DEs - Solve the DE using various methods:
  • Separable DEs - Separate the variables and integrate.
  • Homogeneous DEs - Use a substitution to rewrite the DE in terms of a new variable .
• Numerical Solutions - Approximate the solution using numerical methods like Euler's method.
• Interpreting DEs - Understand the properties of the DE to gain an intuitive understanding of the system.